Penalized Likelihood Regression in Reproducing Kernel Hilbert Spaces with Randomized Covariate Data

Penalized likelihood regression consists of a category of commonly used regularization methods, including regression splines with RKHS penalty and the LASSO. When the observed data comes from a non-Gaussian exponential family distribution, a penalized log-likelihood is commonly used to estimate of the regression function. This technique allows a flexible form of the estimator and aims at an appropriate balance between the goodness-of-fit and the flexibility of the estimator. This thesis is composed of two major parts, both of which are within the framework of penalized likelihood regression. The first part of the thesis presents a direct extension of penalized likelihood regression with RKHS penalty to the situation when the observed covariates are probability spaces. In order to estimate the regression function, we use a penalized likelihood that incorporates the covariate distribution information. We prove that the penalized likelihood estimate exists under a mild condition. In the computation, we propose a dimension reduction technique to minimize the penalized likelihood and derive a GACV (Generalized Approximate Cross Validation) to choose the smoothing parameter. A direct implementation of our methods is to handle incomplete data problems such as covariate measurement error and partially missing covariates.